The exceptionally rapid development of highly flexible, reusable artificial intelligence (AI) models is likely to usher in newfound capabilities in medicine. We propose a new paradigm for medical AI, which we refer to as generalist medical AI (GMAI). GMAI models will be capable of carrying out a diverse set of tasks using very little or no task-specific labelled data. Built through self-supervision on large, diverse datasets, GMAI will flexibly interpret different combinations of medical modalities, including data from imaging, electronic health records, laboratory results, genomics, graphs or medical text. Models will in turn produce expressive outputs such as free-text explanations, spoken recommendations or image annotations that demonstrate advanced medical reasoning abilities. Here we identify a set of high-impact potential applications for GMAI and lay out specific technical capabilities and training datasets necessary to enable them. We expect that GMAI-enabled applications will challenge current strategies for regulating and validating AI devices for medicine and will shift practices associated with the collection of large medical datasets. This review discusses generalist medical artificial intelligence, identifying potential applications and setting out specific technical capabilities and training datasets necessary to enable them, as well as highlighting challenges to its implementation.

A patient is lying on the operating table as the surgical team reaches an impasse. They can't find the intestinal rupture. A surgeon asks aloud, "Check whether we missed a view of any intestinal section in the visual feed of the last 15 minutes." An artificial intelligence medical assistant gets to work reviewing the patient's past scans and highlighting video streams of the procedure in real time. It alerts the team when they've skipped a step in the procedure and reads out relevant medical literature when surgeons encounter a rare anatomical phenomenon.